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evaluation  and  controllers  often  reported  that  the  size  of  the  1”  x  5”  strip  prevented  writing.  The  ability  to  locate  a  particular  strip 
and  find  the  information  on  it  seems  to  suffer  with  a  reduction  in  size  as  tested  in  this  study.  On-line  expert  evaluations  and 
controller  opinions  echoed  this  problem.  Locating  strips  might  have  been  especially  difficult  for  the  R-side,  thus  leading  to  large 
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with  1”  x  5”  strips,  removal  of  deadwood  seems  less  likely  to  be  negatively  affected  by  reduction  in  strip  size.  The  on-line  expert 
evaluation  rated  the  1”  x  5”  strips  negatively  and  the  subject  matter  experts  recorded  more  negative  comments  about  removal  of 
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Executive  Summary 


Currently,  en  route  air  traffic  control  depends 
on  computer-augmented  radar  information  avail¬ 
able  on  the  Plan  View  Display  (PVD),  Computer 
Readout  Device  (CRD),  and  the  flight  informa¬ 
tion  available  on  the  Flight  Progress  Strip  (FPS). 
Thirty-one  fields  for  information  are  printed  on 
the  strips.  This  information  supplements  the 
data  available  from  the  PVD.  While  an  aircraft  is 
in  the  controller’s  sector,  the  controller  writes  on 
the  corresponding  strip  to  reflect  the  control 
instructions,  any  changes  made  in  the  flight  plan, 
and  other  contacts  with  the  aircraft. 

This  report  describes  an  experiment  that 
compared  the  effects  of  using  a  standard-sized 
(VI i6”  x  8”)  FPS  and  a  smaller  (1”  x  5”)  FPS 
containing  less  information  on  the  performance 
and  workload  of  controller  teams.  The  teams, 
from  Minneapolis  ARTCC,  controlled  simulated 
air  traffic  in  a  mixed  radar-nonradar  environ¬ 
ment.  Overall,  the  1”  x  5”  reduced  strip  yielded 
deficits  in  the  control  of  nonradar  flights  but  not 
radar  flights.  This  was  evidenced  in  the  subject 
matter  experts’  evaluation  of  nonradar  separa¬ 
tion,  strip  processing  and  board  management, 
and,  to  a  marginal  extent,  in  the  efficiency  with 
which  traffic  was  moved  through  the  sector.  The 
R-side  controller’s  awareness  was  also  rated  lower 
when  using  the  smaller  strips.  Interestingly,  the 
controllers’  evaluation  of  their  own  performance 
did  not  reflect  this  difference  between  reduced  and 
normal  strips.  This  may  help  explain  why  control¬ 
lers  did  not  compensate  for  the  smaller  strips  to  any 
great  extent. 

Only  the  radar-side  (R-side)  controller  exhibited 
any  compensatory  behaviors  and  only  the  R-side 
reported  increased  workload.  R-side  controllers 
pointed  to  the  PVD  more  often.  Although  there 
was  little  compensatory  activity,  the  R-side  control¬ 
ler  felt  the  workload  was  greater  with  the  smaller 


strips.  The  R-side  controller  felt  it  was  more  effortful 
and  more  frustrating  working  with  the  1  ”x  5”  strips. 
Despite  being  under  a  self-reported  heavier  work¬ 
load,  the  controllers  nevertheless  were  able  to  per¬ 
form  secondary  tasks,  such  as  granting  pilot  requests, 
as  often  and  as  quickly  using  the  smaller  strips  as 
they  did  using  standard  strips. 

This  study  also  supplied  information  about 
the  specific  air  traffic  activities  likely  to  be  af¬ 
fected  by  a  reduction  to  a  1”  x  5”  FPS.  Strip 
marking,  speed  of  strip  processing,  and  some 
aspects  of  board  management  seemed  especially 
affected.  Inferior  strip  marking  was  evidenced  in 
the  on-line  expert  evaluation  and  the  controllers 
often  reported  that  the  size  of  the  1”  x  5”  strip 
prevented  writing.  The  ability  to  locate  a  particu¬ 
lar  strip  and  to  find  the  information  on  the  strip 
once  it  was  found  seems  to  suffer  with  a  reduction 
in  the  size  of  the  strip  used  in  this  study.  The  on¬ 
line  expert  evaluation  and  the  controller  opin¬ 
ions  echoed  this  problem.  Locating  strips  might 
have  been  especially  difficult  for  the  R-side,  thus 
leading  to  the  large  difference  in  self-reported 
frustration.  The  controllers  also  noted  specific 
problems  with  the  strip  display,  including  the  use 
of  shading  to  replace  information  that  is  typically 
presented  in  red.  Of  board  management  respon¬ 
sibilities — considered  by  the  controllers  as  gen¬ 
erally  inferior  with  1”  x  5”  strips — removal  of 
deadwood  seems  less  likely  to  be  negatively  af¬ 
fected  by  reduction  in  strip  size.  The  on-line 
expert  evaluation  rated  the  1”  x  5”  strips  nega¬ 
tively  and  the  subject  matter  experts  recorded 
more  negative  comments  about  removal  of  dead- 
wood  under  that  condition. 

Overall,  this  study  does  not  permit  recommen¬ 
dation  of  the  l”x  5”  reduced  strip  as  designed  for 
this  study.  Suggestions  for  selecting  a  less  than 
standard  size  FPS  are  made  in  this  report. 


Reduced  Flight  Progress  Strips 
in  En  Route  ATC  Mixed  Environments 


Introduction 

The  Federal  Aviation  Administration  (FAA)  has 
been  engaged  in  an  intense  effort  to  modernize  the 
equipment  that  controllers  use  to  help  ensure  the  safe 
and  expeditious  movement  of  the  nation's  air  traffic. 
For  reasons  detailed  elsewhere,  attempts  to  completely 
modernize  the  system  have  met  with  only  partial 
success.  Plans  for  the  Initial  Sector  Suite  System 
(ISSS)  have  been  revised  significantly.  The  state-of- 
the-art  common-consoles  that  were  the  technological 
underpinnings  of  ISSS  are  available  and  will  be  placed 
in  the  field  soon. 

Unfortunately,  the  efforts  to  automate  the  presenta¬ 
tion  of  flight  data  have  been  less  than  successful,  and  the 
common  console  is  no  longer  planned  to  incorporate 
electronic  renditions  of  aircraft  flight  progress  strips. 
Thus,  the  problem  is  to  take  maximum  advantage  of  the 
functional  aspects  of  the  common  console,  while  provid¬ 
ing  controllers  with  the  flight  data  information  they  need 
to  separate  aircraft. 

Currently,  en  route  flight  progress  data  are  presented 
on  1  5/16”  x  8”  strips  of  paper  called  Flight  Progress 
Strips  (FPSs).  The  FPS  includes  31  fields  in  which 
information  about  a  particular  flight  is  presented  by  the 
computer  or  entered  and  modified  by  the  controller.  A 


figure  of  the  current  FPS,  together  with  an  explanation 
of  the  fields,  can  be  found  in  FAA  Order  711 0.65  and  is 
reproduced  in  Figure  1  of  this  report.  The  FPS  is  placed 
in  a  plastic  strip  holder  and  these  holders  are  sequenced 
and  manipulated  in  vertical  bays  next  to  the  radar  scope 
or  plan  view  display  (PVD).  If  the  8”  FPS  is  retained,  a 
bay  of  these  strips  will  quickly  cover  the  radar  associate's 
side  of  the  common-console,  making  access  to  the 
console’s  technology  difficult,  if  not  impossible. 

The  1  ”x  5”  Flight  Progress  Strip 

One  solution  to  this  problem  is  to  present  controllers 
with  flight  progress  data  in  much  the  same  way  that  it  is 
currently  presented,  but  to  reduce  the  size  of  the  FPS  so 
that  more  of  the  new  equipment  is  accessible  by  the 
controller.  However,  the  functionality  of  the  FPS  could 
be  threatened  by  reducing  its  physical  size  or  by  eliminat¬ 
ing  some  of  the  3 1  information  fields.  The  FPS  not  only 
supplies  information  to  the  controller,  it  is  written  on 
and  modified  by  the  controller.  Thus,  a  smaller  strip  and 
its  concomitant  reduction  of  constituent  information 
could  affect  both  the  controller’s  ability  to  locate  and 
extract  information  about  a  flight  and  the  ability  to  find 
enough  space  to  update  and  modify  information  on  the 
strip.  Some  writing  space  can  be  gained  by  eliminating 
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Figure  1.  Examples  of  flight  progress  strips  used  in  this  study.  On  top  is  the  flight  progress  strip  (FPS) 
used  today.  On  the  bottom,  is  the  1”  x  5”  reduced  flight  progress  strip  (RFPS),  which  has  the  call  sign 
and  computer  ID  in  bold. 
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fields  that  are  no  longer  of  value  to  the  controller —  and 
this  was  done  in  the  current  study.  It  is  important  to 
ascertain  whether  controllers  can  effectively  control  traf¬ 
fic  when  the  FPS  is  physically  reduced  in  size  and  some 
of  the  fields  typically  found  on  the  FPS  are  no  longer 
present. 

There  is  some  evidence  that  removal  of  informa¬ 
tion  on  the  FPS  would  not  necessarily  hurt  ATC  perfor¬ 
mance  (Albright,  Truitt,  Barile,  Vortac,  &  Manning, 
1994;  Vortac,  Edwards,  Fuller,  &  Manning,  1993; 
Vortac,  Barile,  Albright,  Truitt,  Manning,  &  Bain, 
1996).  Vortac  and  colleagues  have  shown  that,  at  least 
in  some  types  of  sectors  and  situations,  the  amount  of 
information  on  the  strip  can  be  pared  considerably 
(Vortac  et  al.,  1993;  1996)  or  in  some  cases  research 
suggests  that  the  strip  may  be  eliminated  altogether 
(Albright  et  al.,  in  press).  Although  the  controller  was 
often  forced  to  compensate  for  the  lack  of  an  FPS  (e.g., 
by  doing  a  Flight  Plan  Readout  (FPR))  there  was  not 
necessarily  an  increase  in  workload  or  decrease  in  perfor¬ 
mance.  In  fact,  in  some  cases,  the  controller  was  able  to 
spend  more  time  looking  at  the  PVD  (using  FPRs  in  lieu 
of  strips)  than  was  the  case  with  full  FPSs.  In  another 
case,  with  only  a  one-line  5-field  strip  and  no  permitted 
strip-marking,  some  aspects  of  ATC  were  actually  supe¬ 
rior  to  a  full  FPS  condition  in  which  the  controllers  were 
required  to  mark  the  strips.  Finally,  the  Air  T raffic  Rules 
and  Procedures  Service  has  recently  given  en  route 
facilities  some  latitude  in  the  use  of  strips.  Thus,  it  is  not 
unreasonable  to  assume  that  removal  of  information 
from  the  current  FPS  may  be  a  benign  modification  of 
the  way  flight  data  are  presented. 

One  caveat  to  the  above  argument  is  that,  to  date, 
efforts  to  test  the  limit  of  FPS  information  have  used  only 
sectors  in  which  the  flights  are  continually  tracked  by  the 
radar.  In  mixed  environments,  where  some  flights  are 
never  under  radar  control,  and  where  others  enter  and 
leave  radar  control,  the  role  of  the  FPS  and  its  constituent 
fields  seems  especially  critical. 

Our  team  was  charged  with  determining  the  effects  of 
reducing  the  size  of  the  current  FPS.  We  began  by 
consulting  with  air  traffic  specialists  at  the  FAA  Academy 
and  then  with  specialists  and  National  Air  Traffic  Con¬ 
troller  Association  (NATCA)  representatives  at  the  Min¬ 
neapolis  Air  Route  Traffic  Control  Center  (ZMP).  All 
consultants  were  intimately  familiar  with  controlling 
traffic  in  mixed  environments.  The  purpose  of  these 
consultations  was  to  determine  what  information,  if  any, 
could  be  eliminated  from  the  FPS.  Final  determination 


of  the  information  to  be  presented  on  the  reduced  FPS 
(RFPS)  was  left  to  the  controller  consultants,  who  as  a 
body  tended  to  be  conservative  in  what  information  was 
removed.  For  example,  although  only  one  controller  felt 
that  groundspeed  was  important,  groundspeed  was,  nev¬ 
ertheless,  retained  on  the  RFPS. 

The  blank  RFPS  was  identical  to  the  current  en  route 
FPS  but  was  reduced  proportionately  to  1”  x  5”  dimen¬ 
sions.  Thus,  all  line  demarcations  were  retained  on  the 
RFPS.  However,  a  number  of  fields  in  which  computer¬ 
generated  information  typically  appeared  were  left  blank 
on  the  RFPS.  Specifically,  the  RFPS  no  longer  contained 
the  verification  symbol,  revision  number,  sector  num¬ 
ber,  revised  ground  speed,  strip  number,  previous  fix 
information  (including  estimated  time),  and  next  posted 
fix.  Font  size  was  also  reduced. 

Finally,  we  were  instructed  to  indicate  information 
that  currently  appears  to  the  controller  in  red  (e.g.,  CID, 
special  routing,  and  callsigns)  with  shading.  Although 
from  a  human  factors  point-of-view,  if  shading  reduces 
contrast  too  dramatically  it  is  unlikely  to  be  a  viable 
candidate  for  increasing  the  readability  of  critical  infor¬ 
mation,  the  printer  targeted  to  print  the  RFPS  that  will 
be  used  with  the  common  console  will  be  incapable  of 
printing  color.  Nevertheless,  to  preview  one  finding  of 
the  report,  we  recommend  that  any  monochromatic 
highlighting  of  critical  information  use  a  method  other 
than  shading  (e.g.,  bolding  or  boxing). 

The  1  ”  x  5”  RFPS  with  eliminated  information,  and  with 
shading  replacing  “red  information,”  appears  in  Figure  1. 
The  FPS  for  the  same  flight  is  reproduced  for  comparison. 
The  current  experiment  compared  the  efficiency  and  effec¬ 
tiveness  of  air  traffic  control  using  the  full  FPS  with  the 
efficiency  and  effectiveness  of  the  RFPS. 

Method 

Site 

The  study  was  conducted  at  the  Minneapolis  Center 
(ZMP),  one  of  20  Air  Route  Traffic  Control  Centers 
(ARTCCs)  in  the  United  States.  ZMP  is  responsible  for 
airspace  from  the  Canadian  border  to  northern  Kansas, 
and  from  eastern  Michigan  to  the  western  Dakotas. 
ZMP  has  a  number  of  mixed  sectors:  Air  traffic  is  under 
both  radar  and  nonradar  control.  The  nonradar  volumes 
of  airspace  exist  because  of  a  lack  of  radar  sites  through¬ 
out  the  center’s  airspace  and  not,  for  example,  because  of 
geographical  impediments  to  radar  coverage. 
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Participants 

Twenty  pairs  of  controllers  participated  in  the  study. 
Thirty-nine  were  full  performance  level  (FPL)  control¬ 
lers  and  one  was  a  developmental  (checked-out  for  the 
sector) .  Mean  time  as  an  FPL  at  ZMP  was  7.43  years,  and 
ranged  from  0  to  2 1 .0  years.  The  controllers  had  worked 
in  their  current  area  for  8  years  (range  .58  -  13.25).  All 
teams  except  one  comprised  controllers  who  had  worked 
together  operationally  as  Radar-side  (R-side)  and  Data- 
side  (D-side)  teams  prior  to  the  experiment.  For  the 
experiment,  one  member  of  each  team  served  as  R-side 
and  one  served  as  D-side  for  both  scenarios.  The  partici¬ 
pants  decided  who  would  perform  which  duties. 

Airspace 

Two  sectors  from  ZMP  were  used  in  the  study  (Sector 
1 ,  Area  1 ;  Sector  5,  Area  2).  Both  sectors  have  substantial 
amounts  of  nonradar  traffic,  in  addition  to  flights  under 
radar  control.  The  sectors  were  chosen  in  consultation 
with  our  subject  matter  experts  (SMEs),  one  from  Area 
1  and  one  from  Area  2. 

Ten  teams  were  from  ZMP’s  Area-1;  for  the  purpose 
of  this  experiment,  they  controlled  simulated  traffic  in 
the  Pellston  (Sector  1)  sector. 

The  official  sector  1  description 

Sector  01  works  all  performances  of  aircraft  from  the 
ground  to  12,000  feet  MSL.  At  Pellston  (PLN)  it  is  nonradar 
5,000  feet  and  below.  South  of  PLN  this  increases  to  7000 
Feet  MSL  and  below,  and  to  the  north  of  PLN  this  increases 
to  10,000  feet  MSL  and  below.  The  sector  becomes  active  in 
the  summer  months  with  a  large  amount  of  recreational 
traffic.  The  complexity  of  this  sector  becomes  evident  with  the 
numerous  overlapping  approaches  coupled  with  the  nonradar 
areas.  Controllers  work  with  two  Approach  Controls,  Saginaw 
(MBS)  and  Collins  (APN).  MBS  is  ARTS  equipped  for 
automated  hand-offs  and  flight  plans  are  passed  via  computer. 
APN  requires  manual  coordinations  of  flight  plans  and  manual 
hand-offs.  Sector  01  is  bordered  on  the  north  and  east  by 
Toronto  Center  (YYZ),  to  which  (the  controller)  must  coor¬ 
dinate  manually  both  for  flight  plans  and  radar  hand-offs. 
Sector  01  is  bordered  on  the  south  by  Cleveland  Center,  Flint 
Sector.  Sector  01  is  bordered  on  the  west  by  Minneapolis 
Center  sectors  02  and  03.  (Taken  from  the  facility Vsector 
narrative,  ZMP7220.1). 

Ten  teams  were  from  ZMP’s  Area-2;  for  the  pur¬ 
pose  of  this  experiment,  they  controlled  simulated 
traffic  in  the  Eau  Claire  (Sector  5)  sector. 


The  official  sector  5  description 

Sector  05  is  a  low  altitude  sector  (0  to  23,000)  involved 
in  handling  a  broad  array  of  traffic.  Though  generally  small  in 
area,  sector  05  is  a  fast  moving  sector.  The  two  airports 
generating  the  most  traffic  are  Eau  Claire  (EAU)  and  La 
Crosse  (LSE) .  EAU  has  4 IFR  approaches.  The  BC  Rwy  4  and 
ILS  Rwy  22  are  the  most  commonly  used  and  vectors  can  be 
provided  for  the  final  approach  course  at  the  BC  Rwy  4.  Radar 
coverage  west  of  EAU  is  generally  3000  to  4000  feet  MSL, 
while  east  of  the  VOR  coverage  below  6000  feet  is  marginal  at 
best.  Thirty  to  40  miles  southeast  of  EAU  coverage  does  not 
exist  below  8000  feet.  At  LSE,  there  is  a  part-time  VFR  control 
tower.  The  ILS  Rwy  19,  VOR  Rwy  13,  and  the  VOR  Rwy  36 
approaches  are  most  commonly  used.  Radar  vectors  are  pro¬ 
vided  to  the  VOR  Rwy  13  and  VOR  Rwy  36  final  approach 
course  while  radar  coverage  north  of  LSE  makes  vectoring  for 
the  ILS  Rwy  1 8  approach  very  difficult.  Coverage  below  5000 
feet  north  of  LSE  is  very  limited.  In  addition,  approach 
services  are  provided  for  numerous  small  airports  scattered 
throughout  the  sector.  Sector  05  works  closely  with  Rochester 
(RST)  and  Waterloo  (ALO)  approach  controls  located  in  the 
southwest  corner  of  the  sector.  RST  Approach  owns  8000  feet 
and  down  while  ALO  owns  10,000  feet  and  down.  A  part- 
time  GCA  unit  operates  at  VOK  field  just  east  of  the  sector  05 
boundary.  They  own  10,000  feet  and  down.  There  is  also  a 
part-time  VFR  control  tower  located  at  the  Fort  McCoy 
(CMY)  airport.  Two  restricted  areas  and  2  MOAs  are  used 
within  the  sector.  The  traffic  flow  is  generally  east  to  west 
consisting  of  all  types  of  aircraft.  Sector  05  remains  open  all  of 
the  time.  All  other  sectors  within  the  area  combine  at  Sector 
05.  (Taken  from  the  facility's  sector  narrative,  ZMP7220.1). 

Design 

Each  team  controlled  air  traffic  during  two  scenarios, 
once  using  full  FPSs  and  once  using  the  1”  x  5”  RFPSs. 
Order  of  the  scenarios,  order  of  the  strip-condition  (FPS 
or  RFPS),  and  assignment  of  strip  condition  to  scenario 
were  counterbalanced  across  the  teams,  such  that  each 
scenario  appeared  equally  often  as  first  or  second,  each 
type  of  strip  was  used  equally  often  first  or  second,  and 
each  scenario  was  controlled  using  each  type  of  strip 
equally  often. 

Scenarios 

Scenarios  were  constructed  and  tested  by  the  area 
SME.  Two  50-minute  scenarios,  comparable  in  diffi¬ 
culty  and  realism,  were  constructed  for  each  area.  The 
scenarios  were  intentionally  designed  to  exercise  the  use 
of  the  flight  strips,  sometimes  including  situations  expe¬ 
rienced  in  the  field,  but  at  a  somewhat  higher  rate.  For 
example,  departures  from  small  airports  make  use  of  the 
flight  progress  strips,  but  their  typical  occurrence  in  the 
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field  was  less  frequent  than  in  our  experimental  sce¬ 
narios.  When  asked  about  the  realism  of  the  scenarios, 
53%  found  nothing  unrealistic  about  the  scenarios.  The 
only  differences  between  the  simulated  scenarios  and 
their  typical  live  traffic  mentioned  with  any  frequency 
(>1)  were  busy  small  airports  (9)  and  more  nonradar 
traffic  (5). 

The  scenarios  were  tested  and  modified  by  the  SMEs 
until  they  appeared  comparable  in  difficulty,  realism, 
and  complexity.  Characteristics  of  the  4  scenarios  appear 
in  Table  1. 

On-line  Production  of  Reduced  Strips 

Plastic  strip  holders  for  the  RFPSs  were  created  by 
removing  the  middle  3”  from  the  strip  holders  used  in 
terminal  facilities.  The  strips  used  by  terminal  control¬ 
lers  are  only  1”  in  height,  neatly  fitting  the  requirements 
for  the  current  study.  Similarly,  the  paper  used  to  print 
terminal  strips  was  used  to  print  the  REPS.  The  paper, 
perforated  every  inch,  was  the  same  color  as  current  en 
route  FPSs,  and  was  blank  on  one  side,  allowing  us  to 
create  the  RFPS  on  the  blank  side  of  the  strip.  A  wooden 
strip  bay  with  metal  dowels  was  constructed  to  permit 
the  sequencing  and  manipulation  of  the  smaller  RFPS. 

Because  it  was  not  possible  to  interface  directly  with 
the  HOST  to  produce  the  RFPSs,  we  utilized  a  program 
(i.e.,  STRIPS)  written  by  the  staff  at  ZMP  explicitly  for 
the  purpose  of  quickly  translating  full  FPSs  into  the 
RFPSs.  An  experimenter  sat  next  to  the  HOST’s  printer 
and  removed  FPSs  as  they  were  made  available.  This 
experimenter  identified  the  call  sign  to  another  experi¬ 
menter  operating  the  STRIPS  program.  The  call  sign  was 


used  to  retrieve  the  relevant  strip  from  a  database  of 
potential  strips.  The  experimenters  then  changed  the 
computer  identification  number  (CID),  any  times  (hours 
and  minutes)  on  the  strip,  and  the  flight's  beacon  code. 
Because  the  CID,  time,  and  beacon  code  depended  on 
the  time  period  during  which  the  problem  was  pre¬ 
sented,  it  was  necessary  that  these  pieces  of  information 
be  entered  for  each  execution  of  the  scenario.  For 
reroutings  made  by  controllers  during  the  scenario,  the 
experimenters  either  pulled  up  a  strip  from  the  database, 
anticipating  that  rerouting,  or  modified  an  existing  strip 
in  the  database  to  accommodate  the  route  change.  Ob¬ 
viously,  the  process  of  removing  strips  from  the  HOST 
printer,  translating  them,  and  then  printing  them  on  the 
terminal  forms  took  longer  than  simply  removing  strips 
from  the  HOST  printer.  This  delay  was  reduced  consid¬ 
erably  by  having  the  SMEs  indicate  which  of  the  several 
strips  printed  for  a  particular  flight  was,  in  fact,  one  that 
a  controller  would  use  for  the  flight.  Nevertheless,  it  is 
undeniable  that  controllers  received  strips  more  slowly 
in  the  RFPS  condition  of  this  experiment  than  they 
would  when  a  dedicated  printer  is  interfaced  directly  to 
the  HOST.  Finally,  the  translated  RFPS  was  printed  on 
an  EPSON  laser  printer  and  handed  to  a  ZMP  develop¬ 
mental  who  placed  the  RFPSs  in  holders  and  delivered 
the  strips  (i.e.,  A-side)  for  the  simulations. 

Dependent  Measures 

We  collected  several  dependent  measures  during  the 
course  of  the  experiment.  The  measures  fall  under  4 
categories:  Performance,  Controller  Behaviors,  Work¬ 
load,  and  Controller  Opinion. 


Table  1.  Characteristics  of  the  4  scenarios  used  in  the  experiment. 


Sector  01 
Scenario  A 

Sector  01 
Scenario  B 

Sector  05 
Scenario  A 

Sector  05 
Scenario  B 

Lenqth  of  scenario  (min) 

50 

50 

50 

50 

Total  aircraft 

41 

41 

41 

40 

Overflights  :::::::: 

14 

9 

8 

12 

;  Military  flights  •  Z  '!:■  \  ■ , J  '  §1 '  %  S'  I  '  ,1; 

3 

4 

0 

1 

Departures 

14 

19 

20 

17 

:  Arrivals  =  '  ';1  ( 

13 

18 

17 

15 

Primarily  or  exclusively  nonradar  flights 

6 

7 

11 

9 

Aircraft  needing  manual  coordination 

12 

12 

13 

10 

Communication 

Frequencies 

8 

8 

5 

5 

Other  sectors,  approach  controls,  and  centers 

11 

11 

10 

10 

Active  military  airspaces 

5 

5 

2 

2 

4 


Performance 

Two  instruments  were  used  to  assess  controller  per¬ 
formance.  One  was  a  version  of  the  standard  OJT 
evaluation  form  (FAA  Form  3 1 20-25)  modified  to  focus 
more  precisely  on  strip  activity;  like  an  OJT  evaluation, 
this  On-line  Expert  Evaluation  (OLEE)  was  performed 
by  expert  controllers  while  the  participant  controller  was 
engaged  in  the  scenario.  The  other  was  the  Post-scenario 
analysis  we  developed  previously  to  assess  the  number  of 
remaining  control  actions  (Vortac  et  aL,  1993).  In 
addition,  2  alerts  from  the  controller’s  system  were 
recorded  when  they  occurred. 

On-line  expert  evaluation.  The  SMEs  assessed  control¬ 
ler  performance  “over  the  shoulder”  while  the  team  was 
controlling  traffic  in  the  scenario.  Each  SME  had  the 
OLEE  form,  as  shown  in  Appendix  A.  The  SME  from 
the  relevant  area  of  specialization  always  observed  the  D- 
side,  while  the  SME  from  the  other  area  observed  the  R- 
side.  This  was  for  2  reasons.  First,  an  FPL  from  a  different 
area  can  more  easily  evaluate  mistakes  apparent  on  the 
radar  than  mistakes  on  the  D-side.  Second,  the  focus  of 
the  study  was  on  changes  in  flight  data  utilization, 
suggesting  the  more  knowledgeable  controller  should 
evduate  D-side  performance.  Of  course,  thd  SME  ob¬ 
servers  might  differ  in  how  they  used  the  OLEE  form, 
but  because  scenarios  were  counterbalanced  across  strip 
condition,  such  differences  cannot  affect  the  results 
systematically.  Each  observer  indicated  whether  each 
performance  category  was  Satisfactory,  Needs  Improve¬ 
ment,  or  Unsatisfactory.  Observers  also  kept  a  tally  of  the 
mistakes  controllers  made  within  each  of  the  OLEE 
categories. 

Post  scenario  analysis.  At  the  end  of  each  scenario,  the 
2  SMEs  consulted  in  completing  the  post-scenario  analy¬ 
sis.  The  post-scenario  analysis  is  a  quasi-objective  mea¬ 
sure  of  performance  that  we  developed  for  an  earlier  set 
of  studies.  The  SME  determines,  for  each  flight,  those 
controller  actions  still  remaining  that  will  successfully 
remove  the  flight  from  the  controller’s  sector.  Assuming 
the  same  starting  point,  and  the  same  amount  of  time 
allowed  for  controlling  traffic,  a  condition  that  leaves 
fewer  remaining  actions  can  be  thought  of  as  one  that 
supported  more  efficient  control.  Finally,  because  FPLs 
typically  agree  on  the  actions  necessary  to  transit  an 
aircraft  out  of  the  sector,  we  believe  the  remaining 
actions  measure  makes  a  strong  appeal  to  objectivity. 


Conflict  alerts/MSAWs.  Finally,  we  recorded  2  mea¬ 
sures  that  may  be  indicative  of  performance  difficulties. 
The  current  system  alerts  the  controller  when  it  predicts 
that  loss  of  separation  or  violation  of  altitude  minimums 
is  imminent.  Although  such  alerts  do  not  mean  that 
separation  was  lost,  or  even  that  it  will  ultimately  be  lost, 
a  larger  number  of  such  occurrences  in  one  condition 
compared  with  the  other  may  again  suggest  that  the 
controller  is  performing  less  effectively. 

Controller  Behaviors 

The  behaviors  in  which  controllers  engage  can  pro¬ 
vide  valuable  clues  to  the  potential  effects  of  a  new  system 
on  their  performance.  Some  of  the  behaviors  are  likely  to 
compensate  for  an  inferior  display.  Other  behaviors  are 
relevant  to  performance  and  suggest  poorer  performance 
in  one  situation  than  another.  All  the  behavioral  obser¬ 
vations  described  below  were  obtained  on-line  by  ob¬ 
servers  seated  behind  the  controllers  during  the  scenarios. 
The  experimenters  used  a  tally-sheet,  and,  where  appro¬ 
priate,  a  stopwatch,  to  record  the  relevant  data.  One 
experimenter  observed  the  behavior  of  the  R-side  and 
another  the  behavior  of  the  D-side.  For  those  behaviors 
involving  communication  between  the  controller  and 
pilots,  other  facilities,  or  other  sectors,  the  data  were 
gathered  following  completion  of  the  scenario  from  a 
multitrack  tape  recorder  that  captured  the  radio  and 
telephone  communications  of  the  controllers. 

Compensatory  Behaviors 

In  order  to  maintain  an  acceptable  level  of  perfor¬ 
mance,  controllers  may  need  to  compensate  for  informa¬ 
tion  not  present  on  the  RFPS,  or  not  easily  seen  or 
retrieved  from  the  RFPS.  Such  compensatory  behaviors 
are  not  necessarily  negative,  but  they  are  certainly  infor¬ 
mative.  For  example,  in  the  Albright  et  al.  study  using 
field  controllers  from  Atlanta  ARTCC,  controllers  with¬ 
out  strips  compensated  by  performing  more  flight  plan 
readouts.  Replacing  strip  inspection  with  FPRs  had 
interesting  consequences.  Controllers  could  spend  more 
time  looking  at  the  PVD  when  performing  FPRs,  and 
using  the  quick  action  keyboard  (QAK)  to  obtain  other 
flight  data,  than  when  he  or  she  obtained  those  data  from 
the  strip  bay.  Thus,  compensatory  behaviors  often  painted 
a  complex  picture  that  must  be  evaluated  carefully. 

In  the  current  study,  we  recorded  several  differ¬ 
ent  controller  behaviors  that  could  serve  as  com¬ 
pensatory  mechanisms. 
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FPRs.  If  the  controller  cannot  easily  access  needed 
information  from  the  strips,  he  or  she  may  choose 
instead,  to  request  the  information  from  the  com¬ 
puter  via  the  system’s  quick  action  keyboard  (QAK). 
The  QAK  can  be  used  to  deliver  the  flight  plan  to  a 
small  screen,  the  computer  readout  display  (CRD). 
Such  a  behavior  could  occur  for  either  the  R-side  or 
the  D-side,  and  thus  we  kept  separate  tallies  of  these 
QAK  activities  for  the  two  team  members. 

Time  on  strip  bay.  If  it  is  difficult  to  see  or  find 
information  on  the  smaller  strips,  the  controller  may 
spend  more  time  viewing  the  strip  bay.  For  the  R-side 
controller,  this  of  course  means  that  he  or  she  is 
spending  less  time  monitoring  the  PVD.  Time-on- 
strip-bay  for  the  D-side  is,  on  the  other  hand,  not  a 
compensatory  behavior.  Thus,  we  recorded  this  be¬ 
havior  for  the  R-side  only. 

Time  in  aural R — D  communication.  A  smaller  strip 
may  be  a  less  valuable  aid  in  communicating  between 
the  two  members  of  the  controller  team.  This  deficit 
could  manifest  itself  in  more  R — D  aural  communi¬ 
cation.  For  example,  the  R-side  may  not  be  able  to 
access  the  information  from  the  RFPS  quickly  and 
subsequently,  may  verbally  request  assistance  from 
the  D-side.  Thus,  for  each  scenario,  we  recorded  the 
cumulative  amount  of  time  that  the  team  members 
spoke  to  each  other. 

Pointing.  If  a  smaller  strip  is  difficult  to  locate  by 
one  controller,  it  may  require  the  other  controller  to 
point  to  it.  This  form  of  nonverbal  communication  is 
quite  common,  but  sharp  increases  in  its  occurrence 
could  be  indicative  of  a  problem  in  searching  the  strip 
bay  and  detecting  the  target. 

J -rings.  J-rings  are  multisided  polygons  that  the 
controller  can  place  around  a  flight  displayed  on  the 
PVD.  These  rings  aid  the  controller  in  discerning 
lateral  separation  between  flights  that  are  at  the  same 
altitude.  Disproportionate  use  of  J-rings  in  one  strip 
condition  could  suggest  that  the  controller  is  pushing 
separation  standards  more  in  that  condition  than  in 
the  other. 

Route  displays.  Another  compensatory  operation 
available  to  the  controller  is  displaying  the  route  of  a 
flight  across  the  PVD.  Again,  if  smaller  strips  make 
discerning  routes  more  difficult,  we  may  see  controllers 


choosing  to  obtain  route  information  by  requesting  a 
route  display.  We  recorded  the  number  of  route 
displays  requested,  regardless  of  which  controller  re¬ 
quested  them. 

Coast  tracks.  For  flights  not  under  radar  control,  the 
controller  can  initiate  a  Coast  track  on  the  PVD.  Such 
tracks  can  move  according  to  the  filed  flight  plan  or 
the  controller  can  move  the  data  blocks  manually.  A 
controller  may  choose  to  initiate  more  Coast  tracks 
when  insufficient  information  is  available  from  the 
strips.  We  recorded  the  number  of  Coast  tracks  dis¬ 
played  for  each  scenario. 

Controller  queries.  If  the  controller  cannot  easily 
retrieve  the  information  from  the  strip,  or  if  the  strip 
proves  to  be  an  unsuitable  medium  to  record  impor¬ 
tant  information,  the  controller  may  simply  choose  to 
query  the  pilot  about  the  missing  data.  We  recorded 
all  radio  and  telephone  communications  during  the 
scenarios  and  subsequently  tallied  the  number  of  such 
queries  made  by  the  R-side  and  D-side  controllers. 

Writing.  Finally,  we  recorded  the  number  of  times 
that  a  controller,  either  R-side  or  D-side,  wrote  on  the 
strips.  Writing  on  the  strips  is  a  central  part  of  board 
management,  and  as  we  have  shown  previously,  an 
integral  part  of  the  behavioral  sequences  that  charac¬ 
terize  ATC  (Edwards,  Fuller,  Vortac,  &  Manning,  in 
press;  Vortac,  Edwards,  Jones,  Manning,  &  Rotter, 
1993;  Vortac,  Edwards,  &  Manning,  1994).  How¬ 
ever,  the  effect  of  smaller  strips  on  the  frequency  of 
writing,  while  important,  is  not  easily  predicted.  One 
possibility  is  that,  because  of  the  absence  of  informa¬ 
tion,  controllers  may  choose  to  write  more  on  the 
smaller  strips.  On  the  other  hand,  because  the  strips 
are  physically  much  smaller  and  have  less  room  on 
which  to  write,  the  controllers  may  choose  to  neglect 
or  postpone  strip  marking  that  they  might  otherwise 
perform.  Thus,  although  the  frequency  of  writing  is 
clearly  important  and  worth  observing,  this  measure 
alone  will  not  be  indicative  of  the  viability  of  the 
smaller  strips. 

Workload 

It  may  be  the  case  that  comparable  performance  can 
be  maintained  by  the  controllers,  but  at  a  cost  in  the 
amount  of  effort  the  controller  is  required  to  expend.  We 
assessed  workload  in  2  ways.  One  was  a  relatively  direct 
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query  of  the  controllers  of  their  effort  using  a  standard 
instrument.  The  other  measure  was  a  more  complex,  less 
direct,  but  more  unobtrusive  measure  of  workload. 

TLX.  We  used  a  modified  version  of  the  NASA  TLX 
form.  The  NASA  TLX  (Hart  &  Straveland,  1988)  is  an 
instrument  that  allows  the  assessment  of  several  dimen¬ 
sions  of  the  workload  construct:  mental  demand,  tempo¬ 
ral  demand,  physical  demand,  effort,  frustration,  and 
performance.  Subjects  placed  an  “x”  on  a  line  ranging 
from  “low”  to  “high”  to  indicate  their  perception  of  the 
workload  when  controlling  traffic  with  FPSs  and  RFPSs. 

Granting  requests.  As  part  of  their  job,  controllers  are 
often  asked  to  respond  to  requests  made  by  pilots,  other 
sectors,  and  other  facilities.  Although  the  frequency  with 
which  controllers  grant  such  requests  and  the  time  it 
takes  them  to  grant  these  requests  clearly  involves  more 
than  workload  (e.g.,  the  particular  traffic  situation, 
feasibility  of  the  request),  it  is  clear  that  workload  would 
be  one  factor  influencing  the  frequency  and  speed  with 
which  requests  are  granted.  Thus,  communication  be¬ 
tween  the  controllers  and  the  pilots,  other  sectors,  and 
facilities  were  recorded.  Requests  of  the  controller  were 
identified  and  the  number  of  those  granted  and  the  time 
taken  to  grant  them  were  analyzed. 

Controller  Opinion 

Finally,  the  opinions  of  the  experts  employed  to 
operate  any  new  system  are  a  vital  part  of  its  assessment. 
Thus,  we  interviewed  each  controller  to  determine  over¬ 
all  opinions  of  the  2  strip  formats.  We  queried  them 
using  continuous-line  scales  to  determine  how  well  they 
liked  the  formats,  how  easy  they  were  to  use,  how  useful 
they  were,  the  quality  of  information,  their  perceived 
efficiency  in  controlling  traffic,  and  their  perceived 
efficiency  in  board  management.  Subjects  placed  an  “x” 
on  a  9.6  cm  line  to  indicate  their  opinion  on  each  of  the 
scales.  We  also  included  a  number  of  open-ended  ques¬ 
tions  designed  to  explore  their  particular  concerns,  if 
any,  with  the  reduced  strip,  including  their  opinion  as  to 
how  it  could  be  improved. 

Procedure 

All  teams  of  controllers  were  tested  during  a  2-week 
period  in  January  1995.  After  giving  informed  consent 
and  completing  a  short  biographical  questionnaire,  the 


team  decided  on  who  would  serve  as  R-side.  The  partici¬ 
pants  then  took  their  respective  places  and  were  allowed 
to  view  and  interact  with  the  system  and  the  strips  for  up 
to  5  minutes  before  the  problem  started.  During  that  5- 
minute  period,  the  participants  received  a  short  briefing 
from  the  SME  from  that  area.  The  briefing  included 
idiosyncrasies  of  controlling  simulated  air  traffic  on  the 
DYSIM  and  an  overview  of  current  traffic  parameters, 
including  “hot”  airspaces,  ILS  approaches,  VOR  prob¬ 
lems,  and  so  on.  The  first  of  the  two  50-minute  scenarios 
then  began. 

During  the  scenario,  an  A-side  controller  placed  the 
FPSs  (or  RFPSs)  in  the  strip  holders.  The  primary 
specialist  stood  over  the  shoulder  of  the  D-side  partici¬ 
pant  and  the  other  SME  stood  over  the  shoulder  of  the 
R-side.  Each  SME  completed  an  OLEE  for  the  control¬ 
ler  they  were  observing.  Behind  and  further  to  the  sides 
stood  the  2  observers  responsible  for  recording  R-side 
and  D-side  activity.  An  observer  monitoring  our  audio¬ 
recordings  ofinterfacility  communications  was  plugged- 
in  overhead  and  sat  behind  the  SMEs.  During  the 
reduced  strip  conditions,  a  pair  of  experimenters  sat  off 
to  the  side  next  to  the  HOST  printer  that  produced  the 
FPSs.  FPSs  were  removed  from  the  printer  and  translated 
into  RFPSs. 

At  the  end  of  50  minutes,  the  controllers  completed 
the  modified  TLX  instrument  and  were  given  a  20  to  25- 
minute  break  while  the  second  scenario  was  set  up. 
During  this  time,  the  SMEs  completed  the  post-scenario 
instrument.  When  the  participants  returned,  the  proce¬ 
dure  was  repeated  for  the  second  scenario. 

At  the  end  of  the  second  scenario,  the  participants 
were  separated  and  interviewed  about  their  experiences 
and  opinions,  using  the  post-experimental  question¬ 
naire.  The  participants  were  then  thanked  and  asked  not 
to  discuss  the  details  of  the  study  with  other  controllers 
until  the  study  was  completed. 

Results 

For  the  analyses  in  this  report,  we  chose  to  compare 
the  FPS  condition  with  the  RFPS  condition  using  un¬ 
corrected  Student  t-tests  at  a  test-wise  alpha  of  .05.  We 
made  this  decision  not  only  because  of  the  temporal 
constraints  placed  on  the  production  of  this  report,  but 
also  because  such  a  statistical  strategy  will  be  liberal  in 
rejecting  the  null  hypothesis,  a  bias  not  without  its 
benefits  when  exploring  for  potential  differences. 
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^ ,  Performance 

We  analyzed  2  instruments  to  assess  performance: 
the  Subjective  OLEE  form  and  the  Remaining  Ac¬ 
tions  form.  The  OLEE  form  contained  overall  ratings 
from  the  SMEs  and  a  count  of  the  number  of  negative 
occurrences  or  comments  by  the  SMEs.  Overall,  the 
frequency  of  negative  comments  was  a  more  sensitive 
instrument  than  the  simple  ratings  of  “satisfactory,” 
“needs  improvement,”  and  “unsatisfactory.” 

OLEE  Analysis 

Although  ratings  tended  to  be  less  sensitive  to 
differences  between  FPS  and  RFPS  than  were  fre¬ 
quency  counts,  the  2  sets  of  data  were  quite  clear  in 
their  overall  agreement  that  the  RFPSs  had  negative 
effects.  The  mean  ratings  (Satisfactory  =  1;  Needs 
improvement  =  2;  Unsatisfactory  =  3)  and  the  fre¬ 
quency  of  mistakes  noted  by  the  SMEs  are  shown  in 
Tables  2a  (R-side)  and  2b  (D-side). 

Both  R-side  and  D-side  evaluations  were  poorer  for 
the  RFPS  than  the  FPS  condition  in  the  separation  of 
nonradar  (but  not  radar)  flights,  in  effective  strip 
marking  essential  for  control,  and  in  locating  the 
strips  within  the  bay.  In  addition,  using  the  1”  x  5” 


strip  resulted  in  an  increased  number  of  negative 
comments  about  the  R-side  controller’s  awareness  of 
the  situation.  For  the  D-side,  the  small  strip  not  only 
affected  the  ability  to  locate  the  strip  within  the  bay, 
but  also  the  ability  to  scan  information  on  a  particular 
strip.  Finally,  the  D-side’s  board  management  respon¬ 
sibility  of  removing  deadwood  (but  not  sequencing) 
was  affected  by  the  smaller  strip.  Overall,  the  smaller 
strips  seemed  to  interfere  with  separation  of  nonradar 
traffic  and  with  the  R-side’s  awareness  of  the  situa¬ 
tion.  This  deficit  may  be  due  to  difficulty  in  locating 
the  strip  or  finding  information  on  it,  marking  essen¬ 
tial  information  on  the  strip,  and  removing  strips  for 
flights  no  longer  under  control.  Interestingly,  radar 
control  was  apparently  unaffected  and  there  was  no 
indication  that  nonverbal  communication  between 
the  members  of  the  pair  was  hindered. 

Remaining  Actions 

Results  from  the  Remaining  Actions  form  showed 
no  overall  difference  between  FPSs  and  RFPSs  for  the 
total  number  of  aircraft  still  under  consideration,  or 
for  the  number  of  proposals  remaining  in  the  bay  (See 
Table  3).  We  then  used  the  remaining  actions  data 
from  the  post-scenario  analysis  to  analyze  flights  under 


Table  2a-  OLEE:  Ratings  (left)  and  Frequency  of  Negative  Comments  (right)  for  R-side. 


[RATINGS 

■5151 

T;: 

FREQUENCIES 

HI  u  a 

laMiiy 

siiifiiii 

SEPARATION  .S:: 

n 

,  SEPARATION 

Radar 

1.60 

Radar 

HEE 

i 

Nonradar 

1.90 

Non  radar 

mm 

1.20 

STRIP  MARKING 

■HI  STRIP  MARKING 

Essential  for  control 

2.35 

2.75 

2.18 

Essential  for  control 

11.25 

16.70 

2.66 

Nonessential 

;  1.80 

2.15 

Nonessential 

H23 

STRIP  PROCESSING 
SPEED 

STRIP  PROCESSING  SPEED 

Locating 

1.20 

5.77 

Locating 

Strip  scan 

1.00 

■1 

Strip  scan 

.05 

.20 

ns 

R-D  COMMUNICATION 

R-D  COMMUNICATION 

Nonverbal  communication 

1.00 

1.05 

ns 

Nonverbal  communication 

.85 

AWARENESS 

AWARENESS 

Awareness 

1.15 

mm\ 

Awareness 

.95 

1.80 

2.43 

BOARD  MANAGEMENT 

BOARD  MANAGEMENT 

Removal  of  deadwood^ 

1.25 

1.40 

Removal  of  deadwood 

Effective  sequencing 

1.00 

hesI 

Effective  sequencing 

.30 

.60 

ns 

Italics  =  p  <  .10 

bold  =  p  <  .05 
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RATINGS 

FPS 

RFPS 

FPS 

RFPS 

SEPARATION 

SEPARATION 

Radar 

1.00 

i.oo" 

ns 

Radar 

0.00 

0.00 

ns 

Nonradar 

1.25 

1.80 

2.07 

Nonradar 

.15 

.75 

2.45 

STRIP  MARKING 

STRIP  MARKING 

Essential  for  control 

1.65 

2.15 

2.24 

Essential  for  control 

2.95 

4.95 

2.89 

Nonessential 

1.10 

ns 

Nonessential 

3.20 

.3.00 

ns 

STRIP  PROCESSING  SPEED 

1.00 

1.70 

4.27 

Locating 

.85 

3.75 

6.10 

Strip  scan 

1.00 

1.10 

ns 

Strip  scan 

.10 

<65 

2.24 

R-D  COMMUNICATION 

Nonverbal  communication 

1.00 

1.10 

ns 

BBBBHgl 

.25 

.90 

ns 

AWARENESS 

Awareness 

1.30 

1.40 

ns 

Awareness 

1.20 

1.30 

ns 

BOARD  MANAGEMENT 

BOARD  MANAGEMENT 

Removal  of  deadwood 

1.30 

1.60 

2.04 

Removal  of  deadwood 

3.75 

6.60 

2.86 

Effective  sequencing 

1.15 

1.20 

ns 

Effective  sequencing 

2.00 

1.50 

ns 

Italics  =  p  <  .10 

bold  =  p  <  .05 


Table  3.  Remaining  actions  analysis. 


REMAINING  ACTIONS^ 

FPS 

Total  aircraft 

14.9 

15.15 

Proposals 

3.65 

3.25 

Aircraft  remaining 

7.05 

7.05 

Route  changes 

.90 

1.15 

Altitude  changes  i 

3.40 

3.50 

Approach  clearances 

2.50 

2.55 

Accept  hand-off 

0.00 

0.00 

Initiate  hand-off 

3.95 

3.40 

Communications 

6.951 

6.80 

NONRADAR  T 

Aircraft  remaining 

4.05 

5.10 

Route  changes 

1.25 

1.75 

Altitude  changes 

2.60 

2.95 

Approach  clearances 

1.35 

1.40 

Accept  hand-off 

0.05 

0.00 

Initiate  hand-off 

2.25 

2.55 

Communications 

4.05 

4.90 

Italics  =  p<  .10 

r 

radar  control  and  flights  not  under  radar  control 
separately.  For  flights  under  radar  control,  no  perfor¬ 
mance  differences  were  observed.  On  the  other  hand, 
nonradar  flights  tended  to  be  less  efficiently  con¬ 
trolled  with  RFPSs  than  FPSs,  <(19)  =  1.88,  p  <  .10, 
with  5  flights  remaining  in  the  RFPS  and  only  4 
remaining  in  the  FPS  condition.  This  deficit  seemed 
to  be  divided  across  a  number  of  remaining  actions 
including  route  changes,  altitude  changes,  hand-offs, 
and  communications,  although  none  of  these  con¬ 
stituent  categories  showed  a  statistical  difference.  Thus, 
smaller  strips  affected  nonradar  flights  more  than 
radar  flights,  although  even  here,  the  effects  were 
marginal.  Thus,  what  remained  to  be  accomplished  by 
the  end  of  the  controller’s  50-minute  scenario,  al¬ 
though  not  nearly  as  dramatic  as  the  OLEE  analysis, 
again  favored  the  normal  strips.  Overall,  the  OLEE 
and  the  Remaining  Actions  analysis  raise  concerns 
about  the  ability  of  controllers  to  perform  as  compe¬ 
tently  with  reduced  strips,  especially  in  the  control  of 
nonradar  flights. 
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Alerts 

Conflict  alerts  and  MSAWs  are,  at  best,  gross 
measures  of  controller  performance,  because  they  could 
occur  even  when  no  problem  is  imminent.  Consistent 
with  Table  4,  there  were  no  reliable  differences  in  the 
frequencies  of  these  alerts  between  the  FPS  and  RFPS 
conditions. 


Table  4.  Frequency  of  alerts. 


' ,  - - - 

ppg 

RFPS 

Conflict  alerts 

2.70 

2.41 

MSAWs 

2.30 

2.61 

Compensatory  Behaviors 

Of  the  12  behaviors  we  tallied  from  the  R-side  that 
could  reflect  compensatory  behavior,  including  some 
communications  events  (e.g.,  requests  of  pilots  for 
current  speed),  only  the  frequency  with  which  they 
pointed  to  the  PVD  evidenced  any  difference  between 
RFPSs  and  FPSs,  *(19)  =  2.79:  When  using  reduced 
strips,  the  R-side  pointed  to  the  PVD  48%  more  often 
(6.07  vs.  4.10)  than  he  or  she  otherwise  did.  Notice 
that  pointing  to  the  PVD  more  was  not  because  the 
controller  was  pointing  to  the  strips  less.  Of  the  5 
behaviors  tallied  for  the  D-side,  none  were  signifi¬ 
cantly  affected  by  the  type  of  strip.  The  frequencies  of 
compensatory  behaviors  appear  in  Table  5. 

Workload 

TLX 

Workload,  as  assessed  by  the  modified  TLX,  was 
analyzed  separately  for  R-side  and  D-side.  The  D-side 
controller  reported  experiencing  comparable  work¬ 
load,  regardless  of  whether  the  FPSs  or  RFPSs  were 
used.  For  the  R-side,  however,  the  RFPSs  produced 
greater  self-reported  effort,  *(19)  =  2.17,  and  frustra¬ 
tion,  *(19)  =  2.79. 

Granting  requests 

Another  less  direct,  but  more  objective  measure  of 
workload,  is  the  number  of  pilot  requests  granted  and 
the  time  required  to  grant  them  (see  Table  6). 


Table  5.  Compensatory  behaviors  for  R-side  and 
D-side,  R-D  communication,  and  performance 
related  controller  behaviors. 


BEHAVIORS!  *  * 

. “"I111 . = 

'  :  EPS.  |  RFPS  H 

R-SIDE 

Coast  tracks 

4.00 

4.13 

Flight  Plan  Readout 

1.65 

1.62 

Route  display 

8.00 

8.91 

Point-to-PVD 

4.10 

6.07 

Writing 

174.9 

165.2 

Point-to-Strip 

34.25 

37.36 

Offset  strip 

3.00 

3.86 

View  strips  (sec) 

1057.00 

1091.30 

Vector  line 

5.30 

5.40 

Range  of  PVD 

4.15 

3.07 

Pilot  requests 

19.40 

19.90 

J-rings 

1.30 

1.35 

D-SIDE 

Flight  Plan  Readout 

1.65 

2.16 

Point-to-PVD 

10.80 

11.73 

Point-to-Strip 

25.15 

26.97 

Offset  strip 

12.55 

10.82 

Writing 

147.80 

134.85 

R-D  COMMUNICATION 

Time  (sec) 

193.55 

198.07 

Italics  =  p<  .10 

Bold  =  p  <  .05 

Controllers  granted  the  same  percentage  of  requests, 
and  granted  them  in  approximately  the  same  amount 
of  time.  Although  the  mean  grant  time  for  FPSs 
appears  considerably  longer  than  the  mean  grant  time 
for  the  RFPS,  this  22  second  difference  is  due  prima¬ 
rily  to  one  team’s  exceptionally  long  latency. 

Opinion 

When  the  controllers  were  asked  to  evaluate  the  2  strip 
formats,  the  results  were  quite  clear  (see  T able  7) .  On  the 
6  scales,  the  controllers  showed  a  significant  preference 
for  the  current  FPS  on  each  scale. 

Additional  open-ended  responses  about  the  RFPS 
were  tallied.  Controllers  were  allowed  to  make  as 
many  or  as  few  comments  as  they  wished;  thus,  the 
following  summary,  unless  indicated,  over-represents 
those  controllers  who  offered  more  suggestions.  When 
asked  what  they  liked  about  reduced  strips,  55% 


10 


Table  6.  Two  measures  of  workload.  Direct  TLX 
reports  and  indirect  frequency  and  delay  of 
granting  pilot  requests.  .  _ 


c  tlx 

R-SIDE 

Mental  demand 

75.60 

79.55 

Physical  demand 

59.00 

69.70 

Temporal  demand 

64.75 

71.50 

Effort 

67.20 

77.25 

Frustration 

51.45 

70.25 

Performance 

48.40 

49.50 

D-SIDE 

Mental  demand 

69.55 

76.65 

Physical  demand 

53.70 

62.30 

Temporal  demand 

64.95 

69.20 

Effort 

62.70 

71.55 

Frustration 

49.95 

61.15 

Performance 

55.60 

48.05 

GRANTING  REQUESTS 

Percent  Granted 

94.92 

95.40 

Time  to  grant 

60.27 

38.96 

Bold  =  p  <  .05 

included:  hard  to  scan  (22),  hard  to  read  (16),  CID 
(12),  altitude  (2),  route  (7),  departure/arrival  arrows 
(1),  east-west  bound  indication  (1).  When  explicitly 
asked  if  the  RFPS  board  was  hard  to  scan,  98%  opined 
that  it  was  harder  to  scan  than  the  normal  bay.  Many 
of  the  information  retrieval  concerns  also  revolved 
around  the  use  of  shading,  rather  than  color,  to  indi¬ 
cate  direction  of  flight  and  CID.  Shading  is  not,  ergo¬ 
nomically,  a  good  method  for  highlighting  critical 
information,  and  several  controllers  commented  on 
this  impact  of  shading;  CID  seemed  especially  af¬ 
fected  by  appearing  in  shading,  rather  than  color. 
Only  3  comments  revolved  around  the  excluded  in¬ 
formation,  previous/next  fix.  When  asked  if  there  was 
enough  room  on  the  reduced  FPS,  75%  said  no. 

Comments  about  board  management  included  strip 
marking  problems:  writing  difficult  (1 1),  no  room  for 
altitude  (5),  hard  to  write  on  cocked  strip  (2),  need  a 
fine  tipped  pen  (1),  wrote  less  (1);  and  manipulation 
problems:  hard  to  manipulate  (11),  hard  to  remove 
deadwood  (1).  There  were  also  comments  about  trying 
to  encode  written  information:  hard  to  read  D-side 
writing  (2),  cannot  see  when  R-side’s  hand  is  in  the 
way  (1).  Sixty-five  percent  of  the  controllers  stated  they 
did  not  write  all  required  markings  and  1 0%  were  unable 
to  write  nonmandated  informa¬ 


Table  7.  Controller  opinion  about  strips. 


Bold  =  p  <  .05 

found  nothing.  Negative  comments  were:  hard  to  see, 
had  to  write  smaller,  less  room,  better  than  no  strip  at 
all,  good  excuse  for  not  using  them.  Positive  comments 
were:  fit  in  the  bay,  less  time  to  sequence,  less  space/ room 
used,  didn’t  have  to  reach  as  far,  convenient  for  the  left 
handed  person,  eye-level,  less  clutter,  less  paper. 

When  asked  how  reduced  strips  impaired  perfor¬ 
mance,  the  majority  of  responses  fell  into  comments 
about  information  retrieval  and  about  board  manage¬ 
ment.  Comments  about  information  retrieval 


tion.  Resequencing  was  also 
viewed  as  more  difficult  with 
the  reduced  strip  (40%)  or  the 
same  (48%),  with  12%  of  the 
controllers  not  performing  any 
resequencing. 

When  asked  what  informa¬ 
tion  they  would  remove  from 
the  reduced  FPS,  70%  said  noth¬ 
ing.  Only  shading  was  suggested 
by  more  than  one  controller. 
When  asked  what  information 
they  would  add  to  the  reduced  FPS,  48%  said  nothing. 
Other  comments  were:  color  (13),  previous/nextfix  (8), 
CID-modifications  (4),  nonradar  indicator  (2),  size  of 
arrival/departure  arrow  (3),  move  beacon  code  (2),  larger 
altitude  (2).  Previous  fix/next  fix  information  was  men¬ 
tioned  by  a  significant  minority.  These  concerns  often 
dealt  with  flights  transiting  from  Canada;  for  such 
flights,  information  about  time  appears  nowhere  else  on 
the  strips.  Thus,  when  previous/next  fix  was  removed, 
strips  for  Canadian  flights  had  no  temporal  information. 


p  OPINION 

FPS  ” 

RFPS 

Ease  of  Use 

10.18 

2.45 

14.02 

Usefulness 

9.88 

3.10 

14.38 

Like 

2.66 

1.58 

2.32 

Quality  of  information 

10.28 

5.04 

8.64 

Efficiency  of  control 

8.95 

4.23 

7.27 

Efficiency  of  board  management 

8.73 

3.53 

6.85 
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i  When  asked  to  identify  circumstances  that  needed 
normal  strips,  controllers  indicated:  multiple  altitude 
changes  (10),  approach/departure  indicators  (10), 
nonradar  environment  (8),  preplanning  (8),  holding 
(7),  scanning  (6),  none  (5),  vectoring  arcs  (4),  when 
busy  (4),  and  resolving  conflicts  (3).  Finally,  we  asked 
controllers  if  they  could  safely  control  more,  less,  or 
the  same  amount  of  traffic  with  the  reduced  strips. 
None  projected  an  ability  to  control  more  traffic  and 
60%  believed  they  would  be  able  to  control  less. 

Conclusions  and  Recommendations 

We  compared  the  performance  and  workload  for 
controllers  controlling  air  traffic  in  a  mixed  radar- 
nonradar  environment  using  the  standard  sized  FPS 
(1  V i6  ”  x  8”)  and  an  FPS  reduced  in  both  size  (1”  x  5”) 
and  the  amount  of  information  it  presented.  The 
results  from  20  teams  of  field  controllers  from  Minne¬ 
apolis  center  are  summarized  in  this  report.  Overall, 
the  1”  x  5”  strip  yielded  deficits  in  the  control  of 
nonradar  flights,  but  not  in  the  control  of  radar 
flights.  This  was  evidenced  in  the  subject  matter 
experts'  evaluation  of  nonradar  separation,  strip  pro¬ 
cessing  and  board  management,  and,  to  a  marginal 
extent,  in  the  efficiency  with  which  traffic  was  moved 
through  the  sector  according  to  the  remaining  actions 
analysis.  Furthermore,  the  R-side  controllers'  aware¬ 
ness  was  rated  lower  when  using  the  smaller  strips. 
Because  the  problems  occurred  only  with  nonradar 
flights,  such  measures  as  conflict  or  MSAW  alerts  or 
the  number  of  J-rings  used  would  not  be  expected  to, 
and  did  not,  vary  with  condition. 

Interestingly,  the  controllers'  evaluation  of  their 
own  performance ,  as  indicated  on  the  TLX,  did  not 
reflect  this  difference  between  reduced  and  normal 
strips.  This  may  help  explain  why  controllers  did  not 
compensate  for  the  smaller  strips  to  any  great  extent. 
In  order  to  compensate  for  the  reduced  strips,  control¬ 
lers  made  only  one  reliable  change  in  their  behavior: 
The  R-side  pointed  to  the  PVD  more  often.  This 
behavior  was  not  because  the  R-side  pointed  to  the 
small  strips  less  often.  Perhaps  the  small  strips  engen¬ 
dered  more  nonverbal  communication  overall,  but 
this  nonverbal  communication  was  not  the  type  that 
alerted  our  subject  matter  experts  to  any  problem  in 
intra-team  communication. 

Although  there  was  little  compensatory  activity, 
the  R-side  controller,  but  not  the  D-side  controller, 
felt  the  workload  was  greater  with  the  1 ''  x  5”  strips.  In 


particular,  the  R-side  controller  felt  it  was  more 
effortful  and  more  frustrating  working  with  the  small 
strips.  Despite  being  under  a  perceived  heavier  work¬ 
load,  the  controllers  nevertheless  were  able  to  perform 
secondary  tasks,  like  granting  pilot  requests,  as  often 
and  as  quickly  using  small  strips  as  they  did  using 
standard  strips. 

This  study  also  supplied  information  about  the 
specific  air  traffic  operations  likely  to  be  affected  by  a 
reduction  to  a  1”  x  5”  FPS.  Strip  marking,  strip 
processing,  and  some  aspects  of  board  management 
seemed  especially  affected.  Inferior  strip  marking  was 
evidenced  in  the  OLEE  analysis  and  the  controllers  often 
reported  that  the  size  of  the  strip  prevented  writing. 

The  ability  to  locate  a  particular  strip  and  to  find 
the  information  on  the  strip  once  it  was  found  seems 
to  suffer  with  strip  reduction.  The  OLEE  and  the 
controller  opinions  echoed  this  problem.  Locating 
strips  might  have  been  especially  difficult  for  the  R- 
side,  thus  leading  to  the  large  difference  in  perceived 
frustration.  The  controllers  also  noted  specific  prob¬ 
lems  with  the  strip  display,  including  the  use  of 
shading  to  replace  information  that  is  typically  pre¬ 
sented  in  red. 

Of  board  management  responsibilities — 
considered  by  the  controllers  as  generally  inferior 
with  small  strips  —  removal  of  deadwood  seems  to  be 
most  likely  to  be  affected  by  strip  reduction.  The 
OLEE  evaluation  rated  the  1”  x  5”  strips  negatively, 
and  the  subject  matter  experts  recorded  more  negative 
comments  about  removal  of  deadwood  under  that 
condition.  Because  few  controllers  complained  about 
handling  the  small  strip  holders,  the  deficiency  is 
likely  to  lie  in  an  inability  to  locate  those  strips  ready 
for  removal. 

Overall,  it  seems  clear  that  the  1”  x  5”  strip  tested 
here  was  inferior  to  the  current  FPS,  especially  in  the 
control  of  nonradar  flights.  Thus,  based  on  these  data, 
we  cannot  recommend  that  the  1”  x  5”  strip,  as 
designed  for  this  study,  be  considered  as  a  design 
option.  This,  of  course,  does  not  mean  that  the  cur¬ 
rent  FPS  is  the  only  method  of  presenting  flight  data. 
A  smaller  strip  may  be  feasible.  Toward  this  end,  it 
seems  clear  that  som6  method  of  highlighting  critical 
information,  other  than  shading,  should  be  used. 
Although  several  controllers  offered  suggestions  about 
the  display  characteristics  of  the  reduced  strip,  there 
was  less  concern  about  the  substantive  information 
removed.  However,  previous/next  fix  information 
was  requested  by  a  significant  minority.  Much  of  this 
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concern  was  in  regard  to  flights  transiting  from  Cana¬ 
dian  airspace.  For  these  flights,  important  time  infor¬ 
mation  is  routinely  available  only  in  the  fields  used  for 
fix  postings.  Thus,  while  display  characteristics  should 
undoubtedly  be  changed,  this  study  also  suggests  that,  at 
least  in  some  cases,  valuable  information  was  removed. 

The  problems  with  the  smaller  size  tested  here  were 
both  in  terms  of  finding  relevant  information  quickly 
and  with  writing  on  the  strip.  Obviously,  a  strip  larger 
than  the  test  version  would  alleviate  many  of  the 
problems  found  here,  but  it  is  unclear  what  size  such 
a  strip  should  be,  short  of  using  the  current  FPS.  A 
larger  strip  would  allow  more  space  for  writing  and, 
together  with  optimally  formatted  information,  may 
be  sufficient.  This  might  be  especially  true  once  the 
controller  gains  some  experience  with  smaller  strips; 
the  controllers  in  this  study  obviously  had  had  no 
prior  experience  with  the  1”  x  5”  strip. 

It  is  also  apparent  that  the  reduction  will  have  its 
greatest  impact  on  nonradar  flights.  In  fact,  other 
work  by  this  research  team  has  suggested  that  in 
sectors  under  radar  control,  strip  marking  might  be 
eliminated  completely,  and  the  strip  size  and  informa¬ 
tion  reduced  substantially.  However,  the  current  study 
suggests  that  this  is  not  the  case  with  nonradar  flights. 
Thus,  one  solution  involves  a  change  in  procedures: 
Required  strip  marking  could  be  eliminated  and  con¬ 
trollers  could  be  given  the  option  to  retain  only  those 
strips  needed  to  control  nonradar  flights.  This  proce¬ 
dural  change  could  also  be  combined  with  a  smaller 
strip,  although  it  is  unclear  from  this  study  what  the 
size  and  characteristics  of  the  reduced  strip  should  be. 
This  study  does  indicate,  however,  that  a  1”  x  5”  strip 
with  the  characteristics  described  in  this  report  will  be 
unlikely  to  allow  controllers  to  perform  their  job  at 
the  current  levels  of  efficiency  and  workload. 
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APPENDIX  A 


Sample  On-Line  Expert  Evaluation  Form 


Bl 

■ 

19 

1 

Separation 

Non-radar 

Radar 

■ 

1 

Essential  Srip  Marking 

Essential  for  effective  control 

Mandated  bv  71 10.65 

■ 

■ 

■ 

SDeed 

Difficulty  locatina  striDS 

Slow  strip  scan 

Non-verbal  communications  (done  verbally  which  can  be  done  non-verbatly) 

Awareness 

Board  Management 

Additions  to  active  bav 

Removal  of  deadwood 

Effective  seauencina 

Sample  On-Line  Expert  Evaluation  (OLEE)  form  used  by  subject  matter  experts  (SMEs)  to  rate  the 
effectiveness  of  controller  activities  involving  strip  usage.  SMEs  rated  R-side  activities  separately  from  D- 
side  activities. 

S  =  Satisfactory,  Nl  =  Needs  Improvement,  U  =  Unsatisfactory. 
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